HLT menu development for `12_4_X` (4/N) + migration to `12_4_0` + frozen menu `V1.2` for 2022 [`12_5_X`] #38482

missirol · 2022-06-23T16:00:47Z

PR description:

Development of HLT menus for CMSSW_12_4_X.
Migration of HLT configs to release template CMSSW_12_4_0.
Introduction of V1.2 frozen (GRun) menu to be used for MC production in 12_4_X.

This is the PR for the master branch, which will then be backported to 12_4_X (the latter is the release cycle currently used for HLT-menu development for Run 3).

Includes the integration of the following JIRA tickets.

CMSHLT-2267: [EXO] Update of displaced-ditau trigger
CMSHLT-2287: [TSG] Update of HLTDebug* EventContents with DQMGPUvsCPU output collections
CMSHLT-2304: [MUO,BPH] Change L1T seed of HLT_Mu15, remove HLT_Mu7p5_TkX
CMSHLT-2315: [ECAL] Disable energy-recovery in ECAL RecHit producers
CMSHLT-2326: [EXO] Update of 2 triggers with new displaced-global-muon reconstruction
CMSHLT-2336: [TSG] Disable unclaimed triggers (90 triggers in total)
CMSHLT-2343: [TSG] Add DQMGPUvsCPU stream to HIon and PRef menus
CMSHLT-2344: [BPH] Disabled HLT_DoubleMu2_Jpsi_DoubleTkMu0_Phi_v
CMSHLT-2346: [BPH] Removed *ToCSCS* single-muon 2018-Parking triggers and dataset
CMSHLT-2348: [BPH] Disabled single-muon 2018-Parking triggers (integrated partially; Paths not yet removed nor replaced)
CMSHLT-2353: [L1T] Update of MET L1T seeds

In addition, HLT configs will now use cms.ConditionalTasks instead of cms.Tasks thanks a recent update of ConfDB (V03-04-00). Further details on the reasons behind this change can be found in #36938.

PR validation:

TSG tests.

If this PR is a backport, please specify the original PR and why you need to backport that PR:

N/A

cmsbuild · 2022-06-23T16:06:13Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-38482/30703

This PR adds an extra 1340KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File Configuration/PyReleaseValidation/python/relval_steps.py modified in PR(s): Rawprime workflow with approximated siStrip clusters for heavy ions #38423, New high granularity pixel alignment for the PCL #38449, Update tau embed wf #38266
- File Configuration/PyReleaseValidation/python/upgradeWorkflowComponents.py modified in PR(s): fix VectorHits workflows with PU #38468, Add B physics wfs to release validation #38397, Add run3 tracking low pu era #33532

cmsbuild · 2022-06-23T16:06:38Z

A new Pull Request was created by @missirol (Marino Missiroli) for master.

It involves the following packages:

Configuration/HLT (hlt)
Configuration/PyReleaseValidation (pdmv, upgrade)
HLTrigger/Configuration (hlt)

@Martin-Grunewald, @jordan-martins, @AdrianoDee, @bbilin, @cmsbuild, @missirol, @srimanob, @kskovpen can you please review it and eventually sign? Thanks.
@makortel, @kpedro88, @fabiocos, @Martin-Grunewald, @silviodonato, @trtomei, @beaucero, @slomeo this is something you requested to watch as well.
@perrotta, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

missirol · 2022-06-23T16:09:04Z

enable gpu

missirol · 2022-06-23T16:09:32Z

please test

cmsbuild · 2022-06-24T01:05:28Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-865996/25736/summary.html
COMMIT: 7d1b9f3
CMSSW: CMSSW_12_5_X_2022-06-23-1100/el8_amd64_gcc10
Additional Tests: GPU
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/38482/25736/install.sh to create a dev area with all the needed externals and cmssw changes.

GPU Comparison Summary

Summary:

No significant changes to the logs found
ROOTFileChecks: Some differences in event products or their sizes found
Reco comparison results: 0 differences found in the comparisons
Reco comparison had 1 failed jobs
DQMHistoTests: Total files compared: 4
DQMHistoTests: Total histograms compared: 19864
DQMHistoTests: Total failures: 9
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 19855
DQMHistoTests: Total skipped: 0
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 3 files compared)
Checked 12 log files, 9 edm output root files, 4 DQM output files
TriggerResults: found differences in 3 / 3 workflows

Comparison Summary

Summary:

No significant changes to the logs found
ROOTFileChecks: Some differences in event products or their sizes found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 50
DQMHistoTests: Total histograms compared: 3654169
DQMHistoTests: Total failures: 1847
DQMHistoTests: Total nulls: 10
DQMHistoTests: Total successes: 3652290
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: -10252.140000000001 KiB( 49 files compared)
DQMHistoSizes: changed ( 11634.0,... ): -1706.259 KiB HLT/BPH
DQMHistoSizes: changed ( 11634.0,... ): -2.432 KiB HLT/Filters
Checked 208 log files, 45 edm output root files, 50 DQM output files
TriggerResults: found differences in 4 / 49 workflows

missirol · 2022-06-24T05:20:42Z

@Martin-Grunewald

I noticed that runIntegration.csh does not make use of the Frozen GRun menu. Should that be updated similarly to runOne.csh ? If so, should I update this in this PR (and backport), or in a future one?

Martin-Grunewald · 2022-06-24T06:37:44Z

Yes, it should be added under 'frozen' together with the Fake* in these PRs.

Martin-Grunewald · 2022-06-24T06:39:32Z

... and also under 'all'

missirol · 2022-06-24T06:44:07Z

under 'frozen' together with the Fake*

In runOne, "frozen" does not include the Fake menus (those are only under "fake"). I assume we want to keep the same structure ("frozen"=frozenGRun, "fake"=fake menus).

Martin-Grunewald · 2022-06-24T07:21:49Z

Ah yes, fake for the fakes, frozen for the real non-fake menus such as 2022v12!!

…1.2 for 2022 (branch: 12_5_X)

missirol · 2022-06-24T12:17:50Z

+hlt

outputs of PR tests are in line with changes to the HLT menus, even though some differences are not fully understood by me (see HLT menu development for 12_4_X (4/N) + migration to 12_4_0 + frozen menu V1.2 for 2022 [12_5_X] #38482 (comment))
in particular, it is not clear what causes the differences in GlobalRecHitsV/HCals, and I don't see how HLT changes can induce those diffs (I see nothing to be changed on the HLT side, and I don't really know how those plots are filled) (clarified in HLT menu development for 12_4_X (4/N) + migration to 12_4_0 + frozen menu V1.2 for 2022 [12_5_X] #38482 (comment))

missirol · 2022-06-24T12:20:43Z

@cms-sw/pdmv-l2 @cms-sw/upgrade-l2 , please consider reviewing this PR and its backport. The latter is needed for the upcoming MC production in 12_4_X.

The changes in Configuration/PyReleaseValidation are simple.

missirol · 2022-06-24T14:57:40Z

I think I understand now the differences in GlobalRecHitsV/HCals.

This PR modifies the FEVTDEBUGHLT EventContent adding (among others) new collections of HCAL RecHits, i.e. hltHbherecoLegacy and hltHbherecoFromGPU (the latter is filled only on GPU).

In the wfs in question, this modifies the output of "step2", which is the input to "step3". The histograms in questions are filled in "step3" by the Validation plugin GlobalRecHitsAnalyzer. For HCAL RecHits only, the latter plugin uses all the HcalRecHit collections available ~~in the input file~~ (in the input file + in the current job, i.e. step3):

cmssw/Validation/GlobalRecHits/src/GlobalRecHitsAnalyzer.cc

Line 31 in 0f0c0f3

    
           consumesMany<edm::SortedCollection<HBHERecHit, edm::StrictWeakOrdering<HBHERecHit>>>();

This can explain the differences (more entries in those HCAL plots post-PR, because the EDM input contains more collections of HCAL RecHits). I dare say that HLT makes no guarantee that the collections in FEVTDEBUHLT are mutually exclusive, so the logic of the Validation plugin might have to improved (by the relevant experts).

srimanob · 2022-06-24T15:29:49Z

+Upgrade

From the upgrade side, only HLT key is updated from 2021 to 2022.

perrotta · 2022-06-24T15:32:41Z

urgent

missirol · 2022-06-25T06:53:42Z

@cms-sw/pdmv-l2 , a second ping to review this PR (and its backport). The only change to review is the renaming of HLT:@relval2021 to HLT:@relval2022.

We should start testing these PRs in IBs asap, to exclude unexpected issues in time for the next 12_4_X release early next week.

kskovpen · 2022-06-25T06:57:39Z

+pdmv

cmsbuild · 2022-06-25T06:58:01Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

missirol · 2022-06-25T07:09:05Z

Thanks, @kskovpen . Could you please sign the backport (#38483) as well?

kskovpen · 2022-06-25T07:11:40Z

Sorry - just did!

qliphy · 2022-06-25T07:41:11Z

+1

cmsbuild added this to the CMSSW_12_5_X milestone Jun 23, 2022

cmsbuild added code-checks-pending hlt-pending orp-pending pdmv-pending pending-signatures tests-pending upgrade-pending labels Jun 23, 2022

missirol mentioned this pull request Jun 23, 2022

HLT menu development for 12_4_X (4/N) + migration to 12_4_0 + frozen menu V1.2 for 2022 #38483

Merged

cmsbuild added code-checks-approved and removed code-checks-pending labels Jun 23, 2022

cmsbuild added tests-started and removed tests-pending labels Jun 23, 2022

missirol changed the title ~~HLT menu development for CMSSW_12_4_X (4/N) + migration to 12_4_0 + frozen menu V1.2 for 2022 [12_5_X]~~ HLT menu development for 12_4_X (4/N) + migration to 12_4_0 + frozen menu V1.2 for 2022 [12_5_X] Jun 23, 2022

cmsbuild mentioned this pull request Jun 23, 2022

Rawprime workflow with approximated siStrip clusters for heavy ions #38423

Merged

cmsbuild added tests-approved and removed tests-started labels Jun 24, 2022

HLT menu development for 12_4_X + migration to 12_4_0 + frozen menu V…

443361e

…1.2 for 2022 (branch: 12_5_X)

missirol force-pushed the HLTdevelopment124X_for125X branch from 7d1b9f3 to 443361e Compare June 24, 2022 07:36

cmsbuild removed the tests-approved label Jun 24, 2022

cmsbuild added the tests-approved label Jun 24, 2022

cmsbuild mentioned this pull request Jun 24, 2022

Adding Xi, Chi_c and Chi_b to the matrix for RelVals. #38494

Merged

cmsbuild added hlt-approved and removed hlt-pending labels Jun 24, 2022

cmsbuild added upgrade-approved and removed upgrade-pending labels Jun 24, 2022

cmsbuild added the urgent label Jun 24, 2022

cmsbuild added fully-signed pdmv-approved and removed pending-signatures pdmv-pending labels Jun 25, 2022

cmsbuild added orp-approved and removed orp-pending labels Jun 25, 2022

cmsbuild merged commit a604cb5 into cms-sw:master Jun 25, 2022

missirol deleted the HLTdevelopment124X_for125X branch June 25, 2022 07:46

cmsbuild mentioned this pull request Jun 25, 2022

Adding DeepMET SONIC Producer to CMSSW #37963

Merged

missirol mentioned this pull request Nov 26, 2022

HLT menu development for 12_5_X (2/N) #40155

Merged

This was referenced Aug 31, 2025

on behaviour of OutputModules in unscheduled execution, and its interplay with HLT #36138

Closed

on running unscheduled HLT modules together with other "steps" #36938

Closed

HLT menu development for 12_4_X (4/N) + migration to 12_4_0 + frozen menu V1.2 for 2022 [12_5_X] #38482

HLT menu development for 12_4_X (4/N) + migration to 12_4_0 + frozen menu V1.2 for 2022 [12_5_X] #38482

Uh oh!

Conversation

missirol commented Jun 23, 2022

PR description:

PR validation:

If this PR is a backport, please specify the original PR and why you need to backport that PR:

Uh oh!

cmsbuild commented Jun 23, 2022

Uh oh!

cmsbuild commented Jun 23, 2022

Uh oh!

missirol commented Jun 23, 2022

Uh oh!

missirol commented Jun 23, 2022

Uh oh!

cmsbuild commented Jun 24, 2022

GPU Comparison Summary

Comparison Summary

Uh oh!

missirol commented Jun 24, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Martin-Grunewald commented Jun 24, 2022

Uh oh!

Martin-Grunewald commented Jun 24, 2022

Uh oh!

missirol commented Jun 24, 2022

Uh oh!

Martin-Grunewald commented Jun 24, 2022

Uh oh!

missirol commented Jun 24, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

missirol commented Jun 24, 2022

Uh oh!

missirol commented Jun 24, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

srimanob commented Jun 24, 2022

Uh oh!

perrotta commented Jun 24, 2022

Uh oh!

missirol commented Jun 25, 2022

Uh oh!

kskovpen commented Jun 25, 2022

Uh oh!

cmsbuild commented Jun 25, 2022

Uh oh!

missirol commented Jun 25, 2022

Uh oh!

kskovpen commented Jun 25, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

qliphy commented Jun 25, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

HLT menu development for `12_4_X` (4/N) + migration to `12_4_0` + frozen menu `V1.2` for 2022 [`12_5_X`] #38482

HLT menu development for `12_4_X` (4/N) + migration to `12_4_0` + frozen menu `V1.2` for 2022 [`12_5_X`] #38482

missirol commented Jun 24, 2022 •

edited

Loading

missirol commented Jun 24, 2022 •

edited

Loading

missirol commented Jun 24, 2022 •

edited

Loading

kskovpen commented Jun 25, 2022 •

edited

Loading